1. Field of Invention
This invention relates to a computer system that integrates together, computer handwriting recognition and speech recognition systems, where key elements are under human control. The computer system further relates to means where individuals use the system to capture and record verbal information, which may be made up of consistent or inconsistent messages. The system may convert this information into text and/or graphic data formats. The resulting data may be useful in computer telephony applications or for further computer processing and/or data storage.
2. Description of the Prior Art
Prior art computer based handwriting recognition suffers from problems of imperfect recognition accuracy. Typically, user hand printing can be recognized with a 95-98% accuracy, if the user pre-trains the recognition software program with samples of his/her hand printed alphabet and numbers. Prior art products having handwriting input means such as the Apple Newton that has not obtained success in the market place. Human speech recognition programs must also be pre-trained with a limited number of speech samples, to obtain recognition accuracy of 95-98%. Continuous speech is more difficult to recognize than discrete speech, and, as such, suffers lower recognition accuracy. Speaker independent recognition is more difficult than speaker dependent pre-trained recognition programs. Speaker independent recognition software programs can be pre-trained for common speaker categories, such as English, male, female, southern accent, mid-western, New York, Jersey, Texan, or some combinations thereof. Recognition accuracy and performance of handwriting and speech input have many shortcomings and problems that have prevented them from being a successful computer application in the market place.
Prior art systems also include message recognition systems that use both speech-recognition and handwriting recognition to transform a consistent message to a recognizable massage. However, in this prior art the original massage gathered by each of the recognition systems must be identical. This prior art system is restrictive, limiting it to transcribing functions only. The invention herein over comes that shortcoming by disclosing key elements and functions of a system having several recognition modes and several computing applications, wherein the source message need not be identical or consistent.
The inventions herein consist of combinations of key elements and modifications to existing elements to make new powerful communications systems, with unique computing applications. The invention herein uses the strengths of both the handwriting and speech recognition, along with telephony and other computing applications to make an improved recognition and computing environment. The invention may use the computing power of microprocessors, including multitasking or multiprocessing operating systems to control the operation of the system. The invention may also include digital signal processors (DSP) for fast calculations. The invention herein teaches a computing system running several programs or tasks, running at roughly simultaneous (e.g., real-time or near real-time). User input modes of handwriting (via a pen/stylus input means) and speech recognition (via an audio capture means) work together for new improved computer input results. The two input modes complement each other well and can be combined with other computer means as disclosed below.
An important objective of this new invention is to provide a quick and easy to use system to capture, record and modify speech-recognized text on the fly during (i.e., in near real-time). This objective may be accomplished by combination of computer telephony, pen input and speech input, each running roughly at the same time under control of computer system.
Another important objective is to provide an information system coupled closely with a computer telephony system, where speech information from one or more telephony systems (wire based or wireless) is recognized, annotated, and converted into text and graphics.
Still another objective is to improve computer based verbal information recognition by combining the individual strengths of handwriting recognition with those of speech recognition, resulting in an easy to use system with an overall increase recognition result.
Another objective is to provide a computer data conferencing system with information capture and transcription of conferencing data, converting it into text and/or graphics information, and possibly recording the data for future use.
Another objective is to provide a computer desktop environment having a human in the loop. The system may be comprised of several near real-time processing steps, where the human may be in control of: 1) the system setup, 2) viewing the initial speech recognition results, 3) making decisions on correcting speech recognition errors, and 4) adding additional information not found in the initial speech message.
Still another objective is to provide a system where the user can make certain decisions on the best parameters in setting up the speech and handwriting recognition programs. Such human in the loop decisions may include selecting languages, dictionaries, grammar rules, phonological rules, dialect options, speech accent options, and special user data. These human in the loop selections will greatly improve the overall speech recognition accuracy and usability of the system.
Yet still another objective is provide a system where handwriting and sketched object recognition accuracy is greatly improved by user selection of certain handwriting recognition parameters, either before recognition processing or during the processing. Such selection or selections may include handwriting, stroke or pattern dictionaries (databases). Also, the system may include selection of a general or special linguistic rules databases.